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Executive Summary 


The Smarter Balanced Assessment (SBAC), implemented in California in 2014-15 
as part of the California Assessment of Student Performance and Progress, is designed to 
evaluate students’ levels of college and career readiness. Student scores on the Smarter 
Balanced Assessment are currently used for both accountability and school improvement 
purposes. Aligned with Common Core State Standards for college readiness, student 
performance on the Smarter Balanced Assessment may also predict students’ success 
in college in a manner similar to other commonly used assessments for predicting 
college SUCCESS. 


In this report, we Investigate how well the Smarter Balanced Assessment, high school 
grade point average (HSGPA), and SAT predict first-year college outcomes for students 
enrolled in the California State University (CSU) system and at the University of California 
(UC) campuses. We also report how the relationship between these assessments and early 
college outcomes differs by key student subgroups (race/ethnicity and socioeconomic 
disadvantage). Specifically, we find: 


« The Smarter Balanced Assessment does as well as the SAT in predicting college 
outcomes, specifically first-year college GPA and second-year persistence rates 
for students enrolled in CSU and UC. 


¢ HSGPA is a stronger predictor of first-year college GPA and second-year 
persistence than either SBAC or SAT at CSU. 


e None of the three assessments (HSGPA, SBAC, SAT) are strong predictors of 
second-year persistence at California State University or University of California 
Campuses. 


e Patterns for predicting early college outcomes with HSGPA, SAT, and 
SBAC persist across all racial/ethnic groups, for students identified as 
socioeconomically disadvantaged, and across high schools of varying quality. 


e« There are observable differences in the magnitude of the variation in performance 

explained by the assessments when comparing student subgroups. In particular, 
the three assessments are less predictive of first-year college GPA at CSU and UC 
for low income students than for more advantaged students. 


¢ High school GPA as a predictor of college success results in a much higher 
representation of low income and underrepresented minority students in the 
top of the UC applicant pool, than do SAT or SBAC test scores. Using high 
school GPA with SBAC versus high school GPA with SAT also results in a more 
socioeconomically and racial/ethnically diverse applicant pool. 
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Predicting College Success: 
How Do Different High School Assessments Measure Up? 


In 2014, the state of California implemented the California Assessment of Student 
Performance and Progress (CAASPP) in order to align state assessment and accountability 
policies with the newly adopted Common Core State Standards (CCSS).1 At the heart of the 
new performance and accountability system is the Smarter Balanced Assessment (SBAC). 
The SBAC Is designed to evaluate a student's full range of college and career readiness 
as defined by the CCSS. Three years after the implementation of the SBAC, we can now 
begin to analyze how well the assessments predict college readiness. In addition, we can 
investigate how the SBACs measure up to other commonly used assessments for predicting 
college success. 


In this research brief, we describe early college outcomes for the 2014-15 cohort 
of California 11th graders, the first cohort of 11th grade SBAC takers. Specifically, we explain 
how well the Smarter Balanced Assessment, high school grade point average, and SAT 
predict first-year college outcomes for students enrolled in the California State University 
(CSU) system and at the University of California (UC) campuses.* We also report how the 
relationship between these assessments and early college outcomes differs by key student 
subgroups (race/ethnicity and socioeconomic disadvantage). 


Data Description 


Our analytic sample includes California 11th grade students who took both the 
SBAC and SAT and who subsequently applied and enrolled as first-time freshmen in 
2016-17 at one of the 23 campuses of the CSU system. We use data from the California 
Department of Education (CDE) for all 11th graders who took the SBAC in the 2014-15 
academic year. We match these individual-level 11th grade assessment data to application 
and enrollment data (which include SAT scores) from the CSU Chancellor's Office.* 
In addition, we include individual-level demographic data (available from the 11th grade 
assessment files), high school grade point average (available from the CSU application 
files), and high school characteristics from public-use data (available from the CDE). 


' California adopted the Common Core State Standards in August 2010 and subsequently aligned the state assessments 
in 2014. 
* An earlier version of this report only included data for one University of California campus—Davis. 
5 Using student name, date of birth, gender, and high school attended, we are able to match 82 percent of Fall 2016 CSU 
applicants from California public high schools to the 11th grade assessment data. 
edpolicyinca.org a 
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We also have a similar analytic sample of California 11th graders who took the SBAC 
in 2014-15, who also took the SAT, and who subsequently enrolled as first-time freshmen 
in 2016-17 at one of the UC campuses. Similar to the CSU sample, we use data for all 11th 
graders who took the SBAC in the 2014-15 academic year and match these individual-level 
11th grade assessment data to application and enrollment data from the UC Office of the 
President.’ For this data set, we similarly include individual-level demographic data (available 
from the 11th grade assessment files), high school grade point average (available from the 
UC application files), and high school characteristics from public-use data (available from 
the CDE). 


Measures 


We focus on two early college outcomes: first-year college performance, 
measured by grade point average (GPA) and persistence to year two; for the CSU 
campuses we also include total units accumulated in the first year. The primary predictors 
are high school grade point average (HSGPA), 11th grade SBAC scores in math and 
English language arts/literacy (ELA), and SAT scores (math, reading, and writing). First, we 
adjust for campus differences to account for the host of other experiences students have 
depending on the CSU/UC campus they choose to enroll in, which may also influence 
our outcomes of interest. We also include a control indicator for whether a student is 
identified as socioeconomically disadvantaged in high school (yes/no).° In addition, our 
analysis includes high school quality as measured by the state’s new College/Career 
Indicator (CCl) levels (Very Low, Low, Medium, High, Very High).® Finally, we conduct 
analyses by key individual subgroups: race/ethnicity (five mutually exclusive categories: 
Asian American/Pacific Islander, Black/African American, Latinx, White, and Other) and 
socioeconomic disadvantage; and by school College/Career Indicator (CCI) levels. 
Descriptive statistics on the CSU and UC analytic samples are presented in the Appendix 
Table A1 and A2 respectively.’ 


* Using student name, date of birth, gender, and high school attended, we are able to match 85 percent of Fall 2016 UC 
applicants from California public high schools, to the 11th grade assessment data. 

° Students who meet the definition of socioeconomic disadvantaged (SED) either qualify for the free or reduced price 
school lunch program or do not have a parent who graduated from high school. 

© The College/Career Indicator (CCl) is one of six indicators included in the new California School Dashboard. The CCI 

is first calculated at the student level, where students are labeled as Prepared, Approaching Prepared, or Not Prepared 
based on their 11th grade ELA and Math SBAC Scores, CTE Pathway Completion, AP and IB Exams, Dual Enrollment, and 
A-G Course Completion. A school-level indicator is then determined by considering the proportion of students in the 
graduating cohort that earned a Prepared status. For example, schools with less than 10 percent of their graduating cohort 
earning Prepared receive a Very Low school-level indicator and schools with more than 70 percent of the graduating 
cohort receiving Prepared are considered Very High on the school-level indicator. For more detailed information on the 
College/Career Indicator (CCI), see the description and Technical Guide available from the CDE: https://www.cde.ca.gov/ 
ta/ac/cm/ccical.asp 

7 Univariate correlations are available in the Appendix, Table A3 and A4. 
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Smarter Balanced Assessment. California first implemented the Smarter Balanced 
Assessment in Spring 2015. The Smarter Balanced Assessment includes three major 
components designed to improve teaching and learning: (1) an online library of formative 
assessments for use by teachers, (2) interim assessments for use by schools or districts to 
monitor student progress towards meeting standards, and (3) a summative assessment 
administered annually to determine students’ mastery of college and career readiness 
standards in ELA and math.®? The annual summative assessment is the cornerstone of 
the Smarter Balanced Assessment and is administered to all students statewide in Grades 
3 through 8 and Grade 11, with some exceptions.’? The Smarter Balanced Assessment is 
delivered via computer and includes both a computer-adaptive test and a performance 
task based on the CCSS for ELA and math.“ The computer-adaptive section includes a 
range of item types such as selected response, constructed response, table matching and 
fill-in, graphing, and drag-and-drop. The performance tasks are extended activities that 
measure a student's ability to integrate knowledge and skills across multiple standards—a 
key component of college and career readiness. The estimated time for the 11th grade 
ELA and math tests combined is 7.5 hours.** The cohort included in this analysis is the first 
cohort in California to participate in the 11th grade Smarter Balanced Assessment. 


SAT. The SAT is a college entrance exam accepted or required at nearly all four-year 
colleges and universities in the U.S. It measures students’ college readiness and is validated 
as a reliable predictor of college outcomes, primarily first-year GPA.“ Most students take 
the SAT for the first time during spring of their junior year and a second time during the 
fall of their senior year. In California, 86 percent of CSU applicants and 85 percent of UC 
applicants take the SAT. The format of the SAT for our sample consists of three sections: 
Mathematics, Critical Reading, and Writing (including an essay). Each of the three 
respective sections Is scored on a scale of 200 to 800, with a total score range of 600 to 
2400. The test is administered on paper and is 3.75 hours long. Our analytical sample 
(the first 11th grade Smarter Balanced Assessment takers) were the last cohort to take the 
SAT before its major redesign. 


§ https://www.cde.ca.gov/ta/tg/sa 
° Reliability coefficients for 11th grade Smarter Balanced Assessment in 2014-2015 are .91 for English Language Arts and 
.89 for Math. https://www.cde.ca.gov/ta/tg/ca/documents/caaspp14techrpt.pdf (p. 281). 
10 Students with disabilities who participate in the alternate assessments and English Learners who are in their first 12 months 
of attending a school in the United States are not required to participate. 
“ https://www.cde.ca.gov/ta/tg/sa/sbacsummative.asp 
2 See: https://portal.smarterbalanced.org/library/en/estimated-testing-times.pdf 
3 Details for validity studies provided by College Board: https://research.collegeboard.org/programs/sat/data/validity-studies 
* Reliability coefficeints for the SAT are .92-.93 for Math, .91-.92 for Critical Reading, and .88-.91 for Writing. See: 
http://media.collegeboard.com/digitalServices/pdf/SAT-Test-Characteristics_of_SAT_2011.pdf 
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A redesigned SAT was launched in March 2016.* The new test has only two 
sections: Math and Evidence-Based Reading and Writing. The total score scale range is 
400-1600, with an optional essay scored separately. The tests are still administered on 
paper and students are given 3 hours (plus 50 minutes for the optional essay) to complete 
the exam. Given the new format of the SAT, a follow-up study, similar to the one presented 
here, |S advised. 


Data Analysis 


We follow a standard validity approach to investigate the relationship between the 
11th grade assessments and early college outcomes. Specifically, we fit a series of adjusted 
multiple correlations to examine the strength of the relationship between HSGPA, SBAC, 
and SAT, respectively, and our outcomes of interest, controlling for other variables. These 
multiple correlations tell us how well we can predict early college outcomes on the basis 
of high school performance indicators; for example, how well can we predict first-year 
college GPA on the basis of a test score. We compare the strength of these relationships 
across different specifications, using different predictor and control variables (e.g., HSGPA 
versus SBAC, or SBAC versus SAT, both with and without demographic control variables).*° 


Because applicants with higher HSGPAs or test scores are more likely to be 
admitted to college, our sample of college enrollees does not contain the full range of 
grades and test scores as the census of California 11th grade test takers. It is standard in 
validity studies to adjust for this “restricted range,” which can result in a distortion of the 
true relationship between the predictors (e.g., high school assessments) and outcomes 
(e.g., college GPAs).*”4® We follow the standard practice of the College Board by adjusting 
for the restricted range of both of the assessments we investigate (SBAC and SAT). We 
provide additional information on our methodology in the Appendix. Finally, it is important 
to note that these analyses are entirely correlational, highlighting the association between 
each assessment and early college outcomes, and should not be interpreted as causal. 


5 See: https://collegereadiness.collegeboard.org/pdf/redesigned-sat-pilot-predictive-validity-study-first-look.pdf 

16 We also do this in a multiple regression framework to present R-squared statistics of the variance explained by these 
models. 

Y’ The narrowing of a score range by selection results in an underestimation of the true relationship between the predictor(s) 
and criterion (e.g., first-year GPA). 

18 See: Shaw, Marini, Beard, Shmueli, Young, & Ng, 2016; Mattern, Kobrin, Patterson, Shaw, & Camara, 2009. 
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ji 
Results 
California State University 


Table 1 presents the adjusted (and raw) multiple correlation coefficients of the main 
predictors (HSGPA, SBAC, and SAT) and first-year GPA (Panel A), second-year persistence 
rate (Panel B), and total credits accumulated in the first-year (Panel C) for the CSU analysis. 
Each cell of the table reflects a separate model. Column 1 includes only the respective 
predictor variables without additional controls; we then sequentially add additional 
controls. Column 2 includes the respective predictor variables and controls for enrollment 
at different CSU campuses. Column 3 adds the socioeconomic disadvantage indicator. 
Column 4 includes the high school CCl indicator. 


This table presents several important findings. First, as is expected in such validity 
studies, correlations adjusted for restricted range are higher than raw correlations. Second, 
HSGPA on its own is more strongly correlated to first-year college GPA than either SBAC 
or SAT scores by themselves. Specifically, we see in Column 1 of Panel A that the adjusted 
correlation coefficient of HSGPA and first-year college GPA Is .45, while the adjusted 
correlation of SAT and first-year GPA is .57 and SBAC and first-year GPA Is .37. This finding 
is consistent with similar validity studies by Black, Cortes, and Lincove (2016) and Mattern 
and Patterson (2014). Third, the multiple correlations that include HSGPA show that 
SBAC and SAT are quite similar (if not identical) in their magnitude of association to first- 
year college GPA, (both 48 in Column 1 with no additional controls, and both .49 /.50 
in Column 4 with all of the controls). Finally, we note that there is no meaningful benefit 
of including both SBAC and SAT. The multiple correlation of HSGPA, SAT, and SBAC and 
all controls (Column 4) with first-year GPA is .50, which is basically the same as the value 
when just SAT or SBAC is included with HSGPA."? The strength of the associations we find 
in this analysis is similar to that found in other validity studies; correlation coefficients— 
corrected for restricted range—between admission test scores and college GPA tend to be 
in the 40s and .50s (Shaw et al., 2016; Kuncel & Hezlett, 2007; Mattern & Patterson, 2014). 


For second-year persistence rates (Panel B), magnitudes are much reduced relative 
to first-year GPA, which ts also typical of studies validating admissions tests such as the 
SAT (Mattern & Patterson, 2014). The overall patterns are similar to those found in Panel A; 
HSGPA on its own is more strongly correlated to second-year persistence rates (.22) than 
either SBAC (.19) or SAT (.19) on their own. SBAC and SAT are similar in their magnitude 
of association to second-year persistence rates when HSGPA and other controls are 
included, both at .24 with all of the additional controls (Column 4). Including both SBAC 
and SAT in models predicting first-year persistence does not offer any predictive benefit. 


1° F statistics are available from the authors by request. 
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Table 1. Multiple Correlation Coefficients, Adjusted (Raw) for CSU Analysis 


Panel A: 


First Year GPA (N=36,519) 


HSGPA 45 (.35 46 (.36) 47 (.37) 48 (.38) 
SAT 37 (.28 38 (.29) 38 (.30) 38 (.30) 
SBAC 37 (.28 38 (.29) 38 (.30) 38 (.30) 
HSGPA & SAT 48 (.38 49 (40) 49 (40) 50 (40) 
HSGPA & SBAC 48 (38 49 (.39) 49 (40) 49 (40) 
HSGPA, SAT, & SBAC A9 (.39 50 (40) 50 (41) 50 (41) 
Campus FE N 7, Y Y 
SED N N Y Y 
HS CCI N N N Y 


1 


Panel B: Persistence To Second Year (N=43,791) 


HSGPA 22 (17 23 (.18) 23 (.18) 24 (.19) 
SAT 19 (.14 19 (.15) 19 (.15) .20 (.15) 
SBAC 19 (.15) 20 (.16) 20 (.16) 21 (.17) 
HSGPA & SAT 23 (.18 24 (.19) 24 (.19) 24 (.19) 
HSGPA & SBAC 24 (19 24 (19) 24 (.20) .24 (.20) 
HSGPA, SAT, & SBAC 24 (19 24 (.20) 24 (.20) .24 (.20) 
Campus FE Y Y Y 
SED N N Y v, 
HS CCI N N Y 


iy-Va\ =) & Oam Ke) 1 Ol ali sm sale Ke) Mei (1-1 Ee s K)) 


2 


HSGPA 50 (.39) 57 (46) 58 (48) 59 (49) 
SAT .60 (.51) 62 (.54) 62 (.54) 62 (.54) 
SBAC 55 (46) 59 (.51) 59 (.51) 59 (.51) 
HSGPA & SAT 64 (.55) .66 (.57) .66 (.57) .66 (.57) 
HSGPA & SBAC .60 (.51) .63 (.54) 63 (.55) .64 (.55) 
HSGPA, SAT, & SBAC 64 (.55) .66 (.58) .66 (.58) .66 (.58) 
Campus FE N Y Y Y 
SED N N Y Y 
HS CCI N N N Y 


Notes: We include CSU cam 


(SED), and the California Co 


pus differences in fixed 
lege/Career Readiness 


outcome due to data constraints. 
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effects models, an individual socioeconomic di 


sadvantage indicator 


ndicator (CCl) to proxy for high school quality. Sample sizes differ by 
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The last set of multiple correlations, where total units accumulated in the first 
year of college is the outcome of interest (Panel C), reveal a pattern similar to the other 
outcomes. With this outcome, however, we see an advantage for the SAT in terms of the 
magnitude of the association. Specifically, in Panel C, Column 1, the adjusted correlation 
of SAT and total units in the first year is .60. The adjusted correlation between HSGPA and 
total units in the first year is .50, and between SBAC and total units is 55. When we look 
at each test score with HSGPA, we again note a slight advantage for SAT in predicting 
total units, at .66, compared to SBAC at .64 (Column 4 with full controls). Overall, 
including both SBAC and SAT in addition to HSGPA to predict total first-year units offers no 
substantively meaningful contribution over including just SAT scores and HSGPA, though 
the addition of the SBAC is still statistically significant. 


In Figure 1 we present the R-squared values from multiple regression analyses based 
on the same models presented in Table 1 (Panel A, Columns 1 and 4), because presenting 
our results in terms of the percentage of variation in first-year GPA explained by each of 
the predictors may be more familiar to some readers. For example, looking at the bars on 
he far left (No Controls), we see that HSGPA explains 20 percent of the variation in first- 
year GPA among CSU freshmen, whereas SAT and SBAC each explain about 14 percent 
of the variation in first-year GPA. The bars on the right include controls for CSU campus 
differences, the socioeconomic disadvantage indicator, and high school CCI levels. As a 
result of these additional controls, more of the variance in first-year GPA is being explained, 
but HSGPA remains the dominant predictor and SAT and SBAC explain virtually the same 
overall variation in college GPA. R-squared values for the other models presented in Table 1 
can be obtained by squaring the multiple correlations presented in Table 1. 


oo 


Figure 1. Proportion of Variation in First Year GPA Explained (CSU Analysis) 
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Exploring subgroup differences at CSU. Tables 2 and 3 present results from our 
fully specified models (those that include all controls as in Table 1, Column 4) by key student 
subgroups: race/ethnicity and socioeconomic disadvantage. Though the relationships 
between the assessments and early college outcomes for specific racial/ethnic subgroups 
(Latinx students in particular) are weaker, Our results overall reveal virtually the same pattern 
across all racial/ethnic groups. There are no meaningful differences between SAT and 
SBAC scores in predicting first-year GPA and second-year persistence (i.e., the coefficients 
never differ by more than .02). As with our main results, there are slightly larger differences 
between the two assessments in predicting total units in the first year; for some groups 
SAT is the stronger predictor of this outcome when compared to a model with HSGPA and 
SBAC (but this difference is never larger than .03). Results by socioeconomic disadvantage 
reveal a similar overall pattern of the predictive power of the assessments between groups. 
Here, however, we note more substantial differences in magnitudes, with consistently lower 
associations between high school assessments and first-year outcomes for low income 
students than for more advantaged students. 


Finally, the patterns of association controlling for school quality (as measured by the 
CCl) in Table 4 suggest similar relationships between each of the high school assessments 
and our outcomes of interest, regardless of the quality of the high school. HSGPA and both 
standardized tests are much stronger predictors of first-year college GPA and second-year 
persistence rates for students at the lowest CCI schools, when compared to higher CCl 
schools, which suggests that these assessments help identify college-ready students at 
some of the state’s weakest high schools. 
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Table 2. Multiple Correlation Coefficients, Adjusted (Raw) for CSU Analysis, by Race/Ethnicity 


Panel A: First Year GPA 


All Asian/PI Black/ Latinx White Other 
Eee | (N=7,163) Af Am (N=17,787) | (N=8,754) | (N=1,090) 
(N=1,725) 
HSGPA A8 (.38) 46 (.37 A2 (,35) 40 (31 51 (45) 49 (At 
SAT 38 (30) 38 (28 33 (.27) 32 (.24) 38 (33 40 (34 
SBAC 38 (30) 39 (31 35 (.29) 30 (.23 39 (34 42 (37 
HSGPA & SAT 50 (40) 49 (.40 AA (37) 43 (34 52 (46 51 (43 
HSGPA & SBAC 49 (AO) 49 (40 5 (,38) 42 (,33) 52 (46 52 (45 
HSGPA, SAT, & SBAC 50 (41) 50 (41 AG (39) 43 (34 52 (46 52 (45 


Panel B: Persistence to Second Year 


All Asian/PI Black/ Latinx White Other 
eat tical (N=8,093) AfAm | (N=21,962) | (N=10,240) | (N=1,307) 
(N=2,189) 
HSGPA 24 (19 23 (19 22 (18 22 (17 23 (.21 22 (.21) 
SAT 20 (.15 20 (.16) 20 (17 A7 (13 19 (.17 20 (.19) 
SBAC 21 (17) 20 (.17 21 (18 18 (15 19 (18 21 (.2) 
HSGPA & SAT 24 (19 23 (19 23 (19 23 (18 23 (.21 23 (21) 
HSGPA & SBAC .24 (.20 25 (.20 23 (.20 23 (18 24 (.21 23 (.22) 
HSGPA, SAT, & SBAC 24 (.02 24 (.20 23 (.20 23 (19 24 (21 24 (.22) 


=e: 1 a\=1 Ova Kol e-| MO ali cym stale Ke) ml dl aiam (-t-1 


Race 


aM Asian/PI Black/ Latinx White Other 
SS) | (N=7,838) AfAm | (N=20,695) | (N=9,744) | (N=1,236) 
(N=2,060) 
HSGPA 59 (49) .62 (.50 A3 (.35 A8 (.38 62 (.57 58 (51 
SAT .62 (.54) 67 (.57 A4 (.37) 50 (42) .65 (.60 62 (.56 
SBAC 59 (.51 64 (.54 AS (.37 A7 (40 61 (.57 60 (.54 
HSGPA & SAT .66 (.57 70 (.60 A8 (41 55 (47) 69 (.64 .66 (.59 
HSGPA & SBAC .64 (.55) 68 (.58 A8 (40 53 (44 .66 (61 64 (57 
HSGPA, SAT, & SBAC .66 (.58 70 (64 A9 (42 55 (47) .69 (.64 .66 (.60 
Notes: All correlation coefficients presented include CSU campus differences (fixed effects), hign school CCI levels, and a 
socioeconomic disadvantage (SED) indicator. 
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Table 3. Multiple Correlation Coefficients, Adjusted (Raw) for CSU Analysis, 


by Socioeconomic Disadvantage 


Panel A: First Year GPA 


NU 
(NEI AxT)) NOT SED 


Socioeconomic Disadvantaged Status 


SED 


(N=17,402) (N=19,117) 
HSGPA A8 (.38 AQ (41 A2 (32 
SAT 58 (.30 37 (.29 54 (.24 
SBAC 58 (.30 .37 (.30 33 (.24 
HSGPA & SAT 50 (40 1 (42 A4 (.35) 
HSGPA & SBAC A9 (40 51 (42 A4 (34 
HSGPA, SAT, & SBAC 50 (41) 51 (43 A5 (.35 


Panel B: Persistence to Second Year 


All 
(N=43,791) NOT SED 


Socioeconomic Disadvantaged Status 


SED 


(N=20,153) (N=23,638) 
HSGPA .24 (.19) 22 (19 22 (17 
SAT .20 (.15) 18 (.15 18 (.13 
SBAC 21 (.17) 19 (.16 19 (.15 
HSGPA & SAT 24 (.19) 25 (19 .23 (18 
HSGPA & SBAC .24 (.20) 23 (19 .24 (19 
HSGPA, SAT, & SBAC .24 (.02) 23 (.20 24 (19 


/e-] a\=1 6 Oem Kole-| Ol ali com stale Ke) fl aiam (-r-1 0 


All 
(N=41,573) NOT SED 


Socioeconomic Disadvantaged Status 


SED 


(N=19,269) (N=22,304) 
HSGPA 59 (49 62 (.54) A8 (.37 
SAT 62 (.54 .66 (.58) 51 (42 
SBAC 59 (.51) 62 (.55) A8 (40 
HSGPA & SAT .66 (.57 .69 (.62) 55 (46 
HSGPA & SBAC 64 (.55 .67 (.59) 53 (44 
HSGPA, SAT, & SBAC .66 (58 70 (.62) 55 (46 
Notes: All correlation coefficients presented include CSU campus differences (fixed effects) and high school CCl levels. 
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Table 4. Multiple Correlation Coefficients, Adjusted (Raw) for CSU Analysis, 
by High School CCI Levels 


All 


Panel A: First Year GPA 


School CCI Levels 


(N=36,519) Very Low Low Medium High Very High 

(N=84) (N=6,560) (N=17,281) (N=8,780) (N=3,683) 
HSGPA 48 (.38) 52 (45) A6 (.35) A4 (35 A8 (4) A9 (43) 
SAT .38 (.30) A2 (41) 34 (.26 36 (.28 37 (.29 39 (.32) 
SBAC .38 (30) 41 (40) .33 (.25) 55 (.28 .37 (.30 AO (.34) 
HSGPA & SAT .50 (40) 56 (48) A8 (.38 A6 (.38 AQ (41 51 (44) 
HSGPA & SBAC AQ (40) 54 (46) A8 (.37 A6 (.38 AQ (Al 51 (44) 
HSGPA, SAT, & SBAC .50 (41) 59 (.50) AQ (38 A7 (.39 50 (42 52 (45) 


Panel B: Persistence to Second Year 


rN 


School CCI Levels 


(N=43,791) Very Low Low Medium High Very High 

(N=112) (N=8,200) | (N=20,899) | (N=10,235) | (N=4,160) 

HSGPA 24 (19 35 (.34 .24 (.18) 22 (18 25 (.20) 21 (.20 
SAT 20 (.15 58 (.36 18 (14) 18 (.14 19 (.15) 20 (.18 
SBAC 21 (.17) SESS 21 (.16) L91(15 19 (.16) 20 (.19) 
HSGPA & SAT 24 (19 AO (.37 .25 (.19) 23 (19 .24 (.20) 22 (.20 
HSGPA & SBAC 24 (.20 58 (.36 .26 (.20) 25 (19 .24 (.20) 22 (.20 
HSGPA, SAT, & SBAC 24 (.20 A6 (43 .26 (.20) 25 (19 .24 (.20) 22 (21 


=e: ] a\=1  Oomm Kol e-| MO) ali cym stale Me) gl aiam (-r-1 


All 


School CCI Levels 


(N=41,573) Very Low Low Medium High Very High 

(N=105) (N=7,695) | (N=19,864) | (N=9,735) (N=4,004) 

HSGPA 59 (49) 50 (44) A8 (.38 55 (46 61 (.52 .60 (49) 
SAT 62 (.54) A4 (Al 50 (42 58 (51 64 (.56 63 (.54) 
SBAC 59 (.51) AS (42 A8 (41 55 (47) 61 (.54) 60 (.51) 
HSGPA & SAT .66 (.57) 53 (46) 55 (46 62 (.55 .68 (.60 .67 (.58) 
HSGPA & SBAC 64 (.55) 52 (45 54 (45 .60 (.52 .66 (.58 .65 (.55) 
HSGPA, SAT, & SBAC .66 (.58) 96 (49 56 (47) 63 (.55 .68 (.61 .67 (.58) 


Notes: All correlation coefficients presented include CSU campus differences (fixed effects) and a socioeconomic 


disadvantage (SED) indicator. 
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University of California 


We conducted the same analyses with data from the University of California. 
Results presented in Table 5 are set up in the same way as those presented for the CSU 
outcomes in Table 1. The UC analysis overall reveals stronger associations between 
the predictors and first-year GPA, relative to the CSU sample. We also note stronger 
associations between each of the assessments (SBAC and SAT) and first-year GPA, relative 
to HSGPA. Specifically, in Column 1 we note that the adjusted correlation between HSGPA 
and first-year college GPA is 48, whereas the adjusted correlation between SAT and 
first-year college GPA is .57 and SBAC and first-year college GPA is .51. We also note that 
the SAT has a stronger association with first-year GPA than does the SBAC, even when 
controlling for HSGPA. However, once we control for campus differences, socioeconomic 
disadvantage, and school characteristics, the predictive power of HSGPA, SAT scores, 
and SBAC scores becomes quite similar. The adjusted correlation between HSGPA and 
first-year college GPA is .57, whereas the adjusted correlation between SAT and first- 
year college GPA is .58 and SBAC and first-year college GPA is .54 (Column 4). And 
while the SAT maintains a stronger association with first-year GPA than the SBAC when 
controlling for HSGPA, the difference is reduced to .02. There appears to be no benefit in 
predictive power to including both tests (SBAC and SAT). Specifically, the adjusted multiple 
correlation of HSGPA and SAT in predicting first-year college GPA is identical to the 
multiple correlation that includes SBAC as well. 


Results for second-year persistence (Panel B) reveal the same pattern as Panel A, 
but with weaker magnitudes overall (as we saw with the CSU analysis), and with even 
greater similarities between SAT and SBAC. Specifically, we observe in Panel B Column 1 
that the adjusted correlation coefficient of HSGPA and second-year college persistence at 
UC is .18, whereas the adjusted correlation of SBAC and second-year persistence rate at 
UC is .19, and with SAT it is .20. In models with all the additional controls and adjusted for 
differences across UC campuses (Column 4), SBAC and SAT are again very similar in their 
magnitude of association to second-year persistence rates at UC. Including both SBAC 
and SAT does not offer an improvement in predicting second-year persistence rates at UC, 
versus just including one or the other test (SAT or SBAC). 
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Table 5. Multiple Correlation Coefficients, Adjusted (Raw) for UC Analysis 


Panel A: First Year GPA (N=22,891) 


HSGPA 48 (.34 A8 (34 54 (43) 57 (.47) 
SAT 57 (.50 57 (.50 58 (.50) 58 (.51) 
SBAC 51 (42 52 (43 54 (46) 54 (47) 
HSGPA & SAT 62 (.53 62 (.53 62 (.54) 63 (.54) 
HSGPA & SBAC 58 (47 58 (47 60 (.50) 61 (.51) 
HSGPA, SAT, & SBAC 62 (.53 62 (.53 63 (.54) 63 (.54) 
Campus FE N x i Y 
SED N N Y Y 
HS CCI N N N Y 


1 
HSGPA 18 (.13 18 (14 19 (.15) 21 (.16 
SAT 20 (.16 20 (.17 20 (.17 2117 
SBAC 9: (15) .20 (.16 20 (.16 20 (.17 
HSGPA & SAT 22 (.18 22 (.18 22 (.18 23 (19 
HSGPA & SBAC 22 (17 22 (17 22 (.18 22 (.18 
HSGPA, SAT, & SBAC 22 (18 23 (.18 23 (.18 23 (.19 
Campus FE N Y Y Y 
SED N N Y Y 
HS CCI N N N Y 


Notes: We include UC campus differences in fixed effects models, an individual socioeconomic disadvantage indicator 
(SED), and the California high school College/Career Readiness Indicator (C 
analysis for first-year GPA does not include UC Riverside due to data availab 


ility. 


Cl) to proxy for high school quality. Panel A 
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In Figure 2 we present the proportion of the variance explained (R-squared values) 
in first-year GPA among UC freshmen on the basis of HSGPA, SAT, and SBAC from 
multiple regression analyses, since, for some readers, presenting our results in terms of 
the percentage of the variation in first-year GPA explained by each of the predictors may 
be more familiar. The bars on the far left, with no control variables, highlight that a greater 
proportion of the variation in first-year GPA at UC is explained by the SAT (32 percent), but 
this is attenuated when controlling for HSGPA (to a difference of 4 percentage points), and 
in models on the right that include additional controls (a difference of 3 percentage points 
comparing HSGPA and SAT with HSGPA and SBAC). R-squared values for the other models 
presented can be obtained by squaring the multiple correlations presented in Table 5. 


Figure 2. Proportion of Variation in First Year GPA Explained (UC Analysis) 
50 
40 


38 58 
34 
32 
= .26 
23 
.20 
10 
.00 


No Controls With Controls (CSU Campus, SED, & CCl) 


Proportion of Variance 


Mi HsGPA [SAT [PSBAC BB HSGPASSAT [ HSGPASSBAC  [) HSGPA, SAT, & SBAC 


Exploring subgroup differences at UC. We also conducted analyses by student 
subgroups for the UC sample. Tables 6 through 8 provide results from the fully specified 
models (those that include all controls as in Table 5 Column 4) by key student subgroups: 
race/ethnicity (Table 6), socioeconomic disadvantage (Table 7), and high school 
CCI (Table 8). In general, results reveal similar patterns across all race/ethnicity and 
socioeconomic status groups. The SAT is the stronger predictor of first-year GPA, relative 
to SBAC, but that difference shrinks to a very small magnitude for nearly all subgroups 
in models that include the full set of controls. There is a less consistent pattern for 
predicting second-year persistence rates, with much lower magnitudes across all models 
and subgroups. Importantly, across all sub-groups we note no distinguishable difference 
between the SAT and SBAC in predicting second-year persistence rates across the UC. 
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Table 6. Multiple Correlation Coefficients, Adjusted (Raw) for UC Analysis, by Race/Ethnicity 


Panel A: First Year GPA 


All Asian/PI Latinx White 
Wa) | (N=8,865) Af Am (N=7,312) | (N=5,289) | (N=779) 
(N=646) 
HSGPA 57 (47) 56 (45) 53 (42 43 (.33) 52 (41 54 (43) 
SAT 58 (51) 55 (46) 55 (45) 46 (.39) 50 (42 56 (48) 
SBAC 54 (47) 54 (45) 54 (45 AO (32) AA (36) 50 (41) 
HSGPA & SAT 63 (54) 61 (51) 60 (50 52 (43) 58 (48 61 (52) 
HSGPA & SBAC 61 (.51) 60 (49) 59 (49 AB (.37) 55 (44 57 (47) 
HSGPA, SAT, & SBAC 63 (54) 61 (51) 60 (51) 52 (.43) 58 (48 61 (52) 


Panel B: Persistence to Second Year 


All Asian/PI Latinx White 
7) | (N=10,368) Af Am (N=9,707) | (N=5,959) | (N=903) 
(N=802) 
HSGPA 21 (.16) 18 (.15) 28 (.24) 18 (13 19 (.16 23 (21 
SAT 21 (.17) 19 (.15) 25 (.22) 18 (.14 AT(AS 25 (22 
SBAC .20 (.17) 18 (.15) .26 (.23) 17 (13 A7 (15 25 (.23 
HSGPA & SAT 235 (.19) .20 (.16) .29 (.25) 21 (.16 20 (.17 .26 (.23 
HSGPA & SBAC 22 (.18) .20 (.16) 29 (.25) 20 (.15 20 (.17 .26 (.24) 
HSGPA, SAT, & SBAC 25 (.19) .20 (.17) .29 (.25) 21 (417 20 (.17 27 (.24 
Notes: All correlation coefficients presented include UC campus differences (fixed effects), high school CCl levels, and a 
socioeconomic disadvantage (SED) indicator. Panel A analysis for first-year GPA does not include UC Riverside. 
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Table 7. Multiple Correlation Coefficients, Adjusted (Raw) for UC Analysis, 
by Socioeconomic Disadvantage 


Panel A: First Year GPA 


Socioeconomic Disadvantaged Status 
All 


(N=22,891) NOT SED SED 

(N=13,054) (N=9,837) 

HSGPA 57 (47 54 (42 A4 (.33) 
SAT 58 (.51 93 (45 A8 (40 
SBAC 54 (47 49 (40 43 (.35 
HSGPA & SAT 63 (54 .60 (.50 54 (44 
HSGPA & SBAC 61 (.51 57 (47 51 (40 
HSGPA, SAT, & SBAC .63 (54 .60 (.50 54 (45 


Panel B: Persistence to Second Year 


Socioeconomic Disadvantaged Status 
All 


(NEVIS) NOT SED SED 
(N=15,037) (N=12,702) 
HSGPA 21 (.16) 18 (.15 19 (15 
SAT .21 (.17) 18 (.14 20 (.17 
SBAC .20 (.17) 17 (.14) 19 (.16 
HSGPA & SAT 23 (.19) 20 (.16 .22 (18 
HSGPA & SBAC 22 (18) 20 (.16 22 (17 
HSGPA, SAT, & SBAC .23 (19) 20 (.17 25 (.18 
Notes: All correlation coefficients presented include UC campus differences (fixed effects) and high school CCI levels. 
Panel A analysis for first-year GPA does not include UC Riverside. 


Patterns by school quality (measured by the CCl) in Table 8 suggest a slightly 
stronger SAT association (relative to SBAC), with larger differences for lower CCI schools, 
with first-year college GPA. However, there is no substantive difference between the 
SAT and SBAC in predicting second-year persistence at UC when looking across CCl 
subgroups. 
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Table 8. Multiple Correlation Coefficients, Adjusted (Raw) for UC Analysis, 
by High School CCI Levels 


All 


Panel A: First Year GPA 


School CCI Levels 


(‘eee sch) | Very Low* Low Medium High Very High 

(N=3,184) (N=9,219) (N=5,809) (N=4,640) 
HSGPA 57 (47) AB (.35) A9 (37, 52 (40) 95 (42 
SAT 58 (.51) A5 (.36) 51 (43 52 (43) 57 (47 
SBAC 54 (47) Al (.32) 46 (.37 A9 (40) 04 (44 
HSGPA & SAT 63 (.54) 55 (43) O7 (47 59 (48) 62 (51) 
HSGPA & SBAC 61 (.51) 52 (40) 53 (42) 57 (45) .60 (48 
HSGPA, SAT, & SBAC 63 (54) 55 (44) O7 (47 59 (49) .63 (.51) 


All 


Panel B: Persistence to Second Year 


School CCI Levels 


(‘Eeyy)) | Very Low* Low Medium High Very High 

(N=3,995) (N=11,368) (N=7,031) (N=5,302) 
HSGPA 21 (.16) 18 (.13 18 (.14 18 (.14) 19 (14) 
SAT 21 (17) 19 (.15 19 (.16) 17 (.13) 19 (15 
SBAC 20 (17) 18 (.14) 18 (.15 16 (.13) 18 (.14 
HSGPA & SAT 23 (.19) 21 (.16 21 (17 20 (.16) 21 (17 
HSGPA & SBAC 22 (18) 21 (.16 20 (.16 20 (.16) 21 (.16 
HSGPA, SAT, & SBAC 25 (.19) .21 (.17) 21 (.17 20 (.16) 21 (17 

Notes: *Insufficient sample size to compute for Very Low CCl schools. All correlation coefficients presented include 

UC campus differences (fixed effects), and a socioeconomic disadvantage (SED) indicator. Panel A analysis for first-year 


GPA does not include UC Riverside. 


Predicting the Top 10 Percent of UC Applicant Pool 


Including or excluding a measure of performance in a model does more than 
affect how well we can predict college outcomes. It also affects the ranking of applicants. 
If two students have the same high school GPA, but one performs better on the SAT 
than the other, a model that only includes high school GPA will rank them equally while 
a model that includes high school GPA and SAT scores will rank the student with the 
higher SAT scores above the student with the lower SAT scores. Similarly, if one model 
includes high school GPA and SAT scores while another model includes high school GPA 
and SBAC scores, the first model will advantage students who do well on the SAT, all else 
equal, while the second model advantages students with high SBAC scores. 
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To gauge the effects of including and excluding different measures of 
performance on the demographic make-up of applicants accepted to college we 
estimate each model using ordinary least squares and all enrollees. We then use the 
estimated parameters in each model to predict the outcome for all applicants. In models 
where the outcome of interest Is first year college GPA, this gives the expected first 
year GPA for each applicant. We then rank each student based on their predicted first 
year GPA and then examine the make-up of the top 10 percent of the applicant pool, as 
ranked by each model. We present these predicted models in Figure 3. 


In Figure 3, Panel A we note the percentage of low income students (measured by 
the SED indicator) that would be in the top 10 percent of the UC applicant pool (based 
on first year GPA) in models with different assessment measures. Predicting college 
performance using just high school GPA, we note that nearly a third of the top decile of 
the applicant pool would be low income, when we only include SAT, then 7.9 percent 
would be low income, and when we only use SBAC, we predict 15 percent would be low 
income students. Including high school GPA with SBAC results in 6.8 percentage points 
more low income students in the top of the applicant pool than including high school 
GPA and SAT. 


In Figure 3, Panel B we do the same thing by race/ethnicity. Predicting college 
performance based on high school GPA we note that of the top decile of applicants, 
2.1 percent would be African American and 21.8 percent would be Latinx (two 
underrepresented minority groups at UC); when we only use SAT, only 0.6 percent and 
4.5 percent would be African American and Latinx, respectively; and when we only use 
SBAC, we estimate that 1 percent and 8.7 percent would be African American and Latinx, 
respectively. Including high school GPA with SBAC results in 0.4 percentage points more 
African American students and 4.4 percentage points more Latinx students than including 
high school GPA and SAT. 
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Figure 3. Predicted Top 10 Percent of UC Applicant Pool for First Year GPA 


Panel A: Percent Socioeconomic Disadvantaged 


50 

40 
a 
a 29 
a 30 : 
Dn 
£ 
5 
5 = 15 a 
o 

10 
: i N Zz 
HSGPA SAT SBAC HSGPA HSGPA HSGPA, 
& SAT & SBAC SAT & SBAC 
Panel B: Percent Racial/Ethnic Composition 
1.00 021 .006 010 .006 .010 .007 

> lil 
3 ~~ 80 ae 
< 
£ 
ia 
o 
S .60 
i) 
ra 
> 
a 
e 40 
£ 
o 50 
9 ; 
5 .20 
o 

.00 

HSGPA SAT SBAC HSGPA HSGPA HSGPA, 
& SAT & SBAC SAT & SBAC 


WB Asian/P!| =) White =) Latinx {J} Black/Af Am 


Notes: All estimates account for UC campus differences and are constructed using first year GPA. 


Discussion/Conclusion 


Improving college readiness has been and remains a priority in California’s 
education reform efforts and accountability agenda. Three years after adopting rigorous 
new college readiness assessments aligned with the Common Core State Standards, 
we examine how these new tests perform in predicting college success. More specifically, 
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we investigate the first cohort of 11th grade Smarter Balanced Assessment takers and 

track them into college at both the California State University and University of California 
campuses. We assess how three different high school assessments—GPA, SBAC, and SAT— 
fare in predicting students’ early college outcomes: first-year college GPA, persistence to 
year two, and total units accumulated. 


Our results for the CSU analysis reveal the following: (1) HSGPA is a stronger 
predictor than either standardized test score measure; (2) the SBAC does as well as the 
SAT in predicting college outcomes of CSU students; (3) none of these assessments are 
a strong predictor of college persistence; and (4) the overall pattern of results holds for 
different subgroups (race/ethnicity subgroups, socioeconomic disadvantage, and by high 
school quality). 


For our analysis of UC students, we find that both standardized test scores are 
stronger predictors of first-year college GPA than of second-year persistence. In fact, 
neither of these assessments nor high school GPA is a strong predictor of college 
persistence (probably because 93 percent of all UC students persist to year two). We 
also find that the SAT is a stronger predictor of first-year performance than SBAC, but 
the difference is quite small in models that account for other individual and school 
characteristics. Finally, the overall pattern of results holds for different subgroups (race/ 
ethnicity, socioeconomic disadvantage, and by high school quality). 


There are several factors that policymakers should take into account as they consider 
the use of the Smarter Balanced Assessment (or any other assessment) for predicting 
college outcomes. First, standardized tests explain a relatively small share of early college 
outcomes, especially when controlling for high school GPA and individual and institutional 
characteristics. In addition, our analysis was based on data from the first cohort of Smarter 
Balanced Assessment takers, who had had minimal exposure to the new standards. There 
have been large improvements in both curricular materials and test administration in the 
years since 2014-15. Third, the students in our analyses were the last cohort to take the 
SAT before its substantial redesign. Additional analyses with cohorts of students who have 
taken the new SAT, and also benefited more fully from the shift in curricular standards that 
followed the adoption of the Smarter Balanced Assessment, would provide an essential 
complement to the present study. Finally, these outcomes may be useful for understanding 
performance (as measured by grades) in college, but that is certainly not the only outcome 
policymakers (or society) care about. In particular, we might care a lot about increasing 
access to college, particularly for underrepresented groups, or about the long-term impact 
of sorting students to different types of colleges or within colleges to majors. We might 
also be concerned about the need for developmental coursework once students enroll in 
college, college persistence to and through graduation, and longer term employment and 
economic mobility. 
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Most colleges and universities nevertheless continue to use scores on standardized 
tests to inform their admissions decisions and to assess the academic strengths of 
applicants who come from a broad range of secondary school experiences. Our results 
suggest that Smarter Balanced Assessment scores are no worse (or better) than SAT 
scores at predicting first-year college GPA and persistence rates at CSU, and are only 
slightly weaker at predicting total first-year units. Among UC students, Smarter Balanced 
Assessment scores do not predict first-year GPA and second-year persistence rates as well 
as SAT scores, but the difference in magnitude is trivial. One possible explanation for these 
fundamental similarities is that there is an overlap in the knowledge and skills assessed by 
the Smarter Balanced Assessment and the SAT; the two tests measure much of the same 
information and aptitudes. Recent studies, however, point out important differences in the 
content covered by the two assessments (Achieve, 2018; Assessment Solutions Group, 
2018; HumRRO, 2016).7° A second and closely related reason is that the Smarter Balanced 
Assessment has been designed to reflect the state’s new effort to align K-12 standards to 
expectations for postsecondary success. 


There are several important caveats to any effort to directly compare the Smarter 
Balanced Assessment to the SAT. First, there are important differences in test format 
between the SAT and Smarter Balanced Assessment, which were described earlier in 
this report. Second, the Smarter Balanced Assessment has—or at least is perceived to 
have—much lower stakes for individual students than the SAT; students taking the Smarter 
Balanced Assessment are not generally under the impression that their performance on 
the test will be considered when they apply to college. Third, students may not retake 
the Smarter Balanced Assessment, which is common with the SAT. Finally, some students 
take preparatory courses to improve their scores on the SAT. For now it is unclear how 
and whether these differences (e.g., low stakes, retakes, and test preparation) between 
the Smarter Balanced Assessment and SAT might bias the results presented here, in either 
direction. 


Our future work will continue to assess the use and effectiveness of various K-12 
assessments in predicting a host of student outcomes, in college and after. As California 
seeks to strengthen alignment between K-12 and postsecondary schooling, our evaluation 
of the standards taught and tested in the K-12 years should provide key information to 
evaluate college and labor market readiness and success. 


20 While the Smarter Balanced Assessment is specifically aligned to the Common Core State Standards for the purpose of 
measuring student achievement of the standards, recent studies suggest the SAT is not fully aligned with Common Core 
State Standards (Assessment Solutions Group, 2018; HumRRO, 2016). 

+t California was the first state to use the 11th grade assessment as an opportunity to give students a signal about their 
college readiness, which it began with the Early Assessment Program in 2004 and which it continues with the Smarter 
Balanced Assessment today. Thus, students’ results on these tests do matter—at least for placement out of developmental 
coursework—though they are not perceived to matter as much as the SAT, which is more broadly used for admissions. 
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Appendix A: Data 


Table A1. Summary Statistics, CSU Analytic Sample, Mean Values [Standard Deviations] 


CSU Applicants Coss] OM Sal ce) (15 
(NEw i hz) (N=36,519) 
Female 0.5808 0.5942 
4934 [4911 
Asian/Pacific Islander 0.1995 0.1961 
3996 [.3971 
Black/African American 0.0534 0.0472 
2248  AZAL 
Latinx 0.4879 0.4871 
A999 4998 
White 0.2279 0.2397 
4195 A269 
Other 0.0313 0.0298 
1742 1702 
English Learner OOS 0.0639 
DSS 2446 
Socioeconomic Disadvantage 0.5346 0.5235 
A988 4995 
CCl = Very Low 0.0027 0.0023 
0515 0479 
CCI = Low 0.1862 0.1796 
3893 3839 
CCI = Medium 0.4597 0.4732 
4984 4993 
CCI = High 0.232 0.2404 
4221 4273 
CCl = Very High 0.1151 0.1009 
3192 3011 
HSGPA 3.356 3.415 
6244 481 
SAT Reading 492.6 487.7 
103.4 90.5 
SAT Math 505.7 502.1 
[110.1 95.7 
SAT Writing 488.3 481.5 
[102.3] 86.8 
SBAC ELA 2651 2653 
[83.23] [75.29] 
SBAC Math 2631 2631 
[101.8] [90.8] 
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Table A2. Summary Statistics, UC Analytic Sample, Mean Values [Standard Deviations] 


UC Applicants UC Enrollees 
(N=71,552) (Evy Aake)) 

Female 0.5786 0.5833 
A938 [493] 

Asian/Pacific Islander 0.2942 0.3739 
A557 4839 

Black/African American 0.0427 0.0289 
2022 1676 

Latinx 0.3953 0.3499 
A889 A769 

White O2ESS 0.2147 
[4231] 4106 

Other 0.0343 0.0326 
[.182] ATS 

English Learner 0.0533 0.0389 
2246 1934 

Socioeconomic Disadvantage 0467 0.4578 
A989 A982 

CCl = Very Low 0.0023 0.0016 
0475 0394 

CCI = Low 0.1518 0.144 
3589 [.3511 

CCl = Medium 0.4162 0.4097 
A929 [4918 

CCI = High 0.25 0.2536 
[.433] [4351 

CCI = Very High 0.1797 0.1912 
[.384] 5932 

HSGPA 3.454 3.642 
[4041] 3002 

SAT Reading 543 576.2 
[109.7] 100.2 

SAT Math 562.5 600.8 
[115.8] 105.4 
SAT Writing 542.8 578.8 
(111.4) 102.7 
SBAC ELA 2686 2709 
[77.71] 67.59 
SBAC Math 2682 2714 
[99.8] [88.90] 
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Table A3. CSU Univariate Correlation Matrix for High School Assessment Measures 


Panel A: CSU Applicants (N=118,210) 


SATm SATW SBACela SBACm Asian ETel4 Hispanic 
& Pl 


HSGPA 1.0000 

SATv 4618 1.0000 

sATm A912 7663 1.0000 

SATW A931 8450 7685 1.0000 

SBACela 4512 6997 5999 6614 1.0000 

SBACm 4987 6568 8204 6498 6598 1.0000 

Asian 1357 1749 .2906 .2040 1552 2582 1.0000 

& Pl 

Black -.1114 -.0836 -.1272 -.0983 -.1148 -.1378 -.1185 1.0000 

Hispanic -.1812 -.3638 -.3919 -.3598 -.2245 -.3016 -4872 -.2318 1.0000 

White 1377 .2830 2343 2624 1625 1686 -.2712 -.1290 -.5303 1.0000 
SED -.1647 -4047 -.3741 -.3975 -.2524 -.2771 -.1131 0232 A506 - 4006 1.0000 


Ly- Val =) = Hn ORSON tal co) | (-1-t-(\ EK Looe he) 


SATW SBACela SBACm PNSETa) Hispanic 
& Pl 

HSGPA 1.0000 
SATv 3201 1.0000 
SATm 5624 7030 1.0000 
SATW 3584 8034 6924 1.0000 
SBACela 3368 6527 5268 6082 1.0000 
SBACm 3923 5/52 7883 5613 5909 1.0000 
Asian 0515 0734 2023 0915 .0888 1920 1.0000 
& Pl 
Black -.0982 -.0589 -.1126 -.0793 -.0951 -.1227 -.1100 1.0000 
Hispanic -.1283 -.3310 -.3614 -.3208 -.1908 -.2708 -4813 -.2170 1.0000 
White 1434 3208 2658 53060 1700 1804 -.2774 -.1250 -.5472 1.0000 
SED 21370 -4078 -.3702 -.3940 -.2444 -.2633 -.0806 0132 4393 -4080 1.0000 
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Table A4. UC Univariate Correlation Matrix for High School Assessment Measures 


Panel A: UC Applicants (N=71,552) 


SATW SBACela SBACm Asian Black Hispanic 
& Pl 

HSGPA 1.0000 
SATVv A207 1.0000 
SATm A495 7633 1.0000 
SATW 4499 8402 7753 1.0000 
SBACela 4034 6791 5776 6372 1.0000 
SBACm 4649 6689 8299 6642 6495 1.0000 
Asian 1149 2212 3556 2672 1714 .2999 1.0000 
& Pl 
Black -.1010 -.0883 -.1367 -.1035 -.1013 -.1396 -.1364 1.0000 
Hispanic -.1972 -4169 -4616 -4237 -.2662 -.3674 -.5220 -.1708 1.0000 
White 1429 2591 1954 2297 1554 1520 -.3563 -.1166 -4463 1.0000 
SED -.1936 -A772. -4520 -4773 -.3002 -.3528 -.1583 0361 4734 -.3532 1.0000 


F-Ya\ =| = $n 0) Om tal ce) | (1-3) PAV a he) 


SATW SBACela SBACm PNSETa) Hispanic White 
& Pl 
HSGPA 1.0000 
SATv 3049 1.0000 
SATm 3270 7083 1.0000 
SATW 6825 8038 7334 1.0000 
SBACela 3036 6107 4833 55/1 1.0000 
SBACm 3550 6061 8023 6027 5821 1.0000 
Asian 0794 2247 3910 .2808 1530 3182 1.0000 
& Pl 
Black -.0654 -.0674 -.1148 -.0845 -.0800 - 1131 -.1334 1.0000 
Hispanic -.1728 -4206 -4958 -4400 -.2466 -.3856 -.5669 -.1266 1.0000 
White 1248 2273 1445 1961 1238 1046 -4041 -.0903 -.3836 1.0000 
SED -.1526 -4820 -4570 -4903 -.2779 -.3402 -.1696 .0306 A696 -.3179 1.0000 
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Appendix B: Methods 


In this study, we estimate the multiple correlation coefficient, R, for assorted sets of 
explanatory variables and outcomes in order to observe the change in predictive power as 
different assessments of interest are included and excluded. Specifically, we calculate: 


R=VR = (P,P2P,)* 


where P,,, P,,, and P,, are partitions of the correlation matrix P such that: 


Pa P 
P = 0x >| 
Pye Py 


We calculate separate correlation matrices for each outcome (y) of interest, Le., 
first-year GPA, persistence to the second year, and units accumulated in the first year. 
Further, separate correlation matrices are calculated depending on what is included in the 
set of predictive (x) variables, the sparsest set of predictive variables being only high school 
GPA, while the fullest set being high school GPA, both SBAC scores, all three SAT scores, 
dummies for demographics, dummies for the College/Career Indicator level of a student's 
high school, and dummies for CSU or UC campuses. 


We can only observe outcomes for the students who are admitted and choose to 
attend the schools in our sample. As students are admitted to schools directly on the basis 
of high school grades and SAT scores and indirectly on the basis of SBAC scores (which 
are highly correlated with grades and SAT scores), our estimates will be biased due to 
selection. This is known as the restriction of range problem as we are much less likely to 
observe outcomes for the students with certain test scores and grades. For an extended 
discussion of why restriction of range causes problems in estimation, see Gulliksen (1950). 
What follows is a brief illustration to develop intuition of the problem and the solution we 
use here. 


In Figure B1, we plot two hypothetical tests, Test 1 and Test 2, against an outcome. 
We note that both tests are positively correlated with the outcome, and that, on average, 
a student who receives some score on Test 1 would be expected to do the same as a 
student who receive the same score on Test 2. However, we can see that the outcome 
values for the students who took Test 1 are much more tightly clustered than the outcome 
values for students who took Test 2. This is the visual indication that Test 1 is much more 
highly correlated with the outcome than Test 2, (i.e., that knowing what a student receives 
on Test 1 will better predict the student's outcome than Knowing what a student receives 
on Test 2). 
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Figure B1. High vs. Low Correlation 


Outcome 


| | | | | | 
0 20 40 60 80 100 


Test Score 


@ Test 1: High Correlation [J Test 2: Low Correlation 


Now imagine that we were to examine how closely related one test score was to 
an outcome when we could observe the outcome for all students regardless of how they 
scored on the test. In the context of this study, this would be analogous to all high school 
students in California who took the SAT then attending CSU. We can see what this might 
look like in Figure B2. If instead we were only able to observe the outcomes of students 
who scored above a certain score on the test, for example because a student might need 
a certain SAT score in order to be admitted to college, then our data might look more like 
what is presented in Figure BS. Even though for test scores over 50 these are the exact 
same points of data as In Figure B2, we now observe how much less tightly clustered the 
data points seem by excluding all outcomes for students with test scores below 50. 
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Figure B2. Outcomes Observed 
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We correct for this bias due to this restriction of range using the method first 
proposed by Pearson (1903); developed in Lawley (1943); and first outlined in Gulliksen 
(1950), then in Lord, Novick, and Birnbaum (1968), as well as in Lewis (2006). Following 
Lewis's notation, let x and y be vectors of random variables with covariance matrix ¥ for 
the full population”. Partition % such that: 


bee 3. 
EE 
yx Dy 
Since y is not observed for the full population, ¥.,, ¥y. and ¥,,, cannot be estimated 
directly from the data. However, if we let s be a selection variable such that s, = 1 if an 
individual from the full population is included in the selected population while s; = 0 if the 
individual is not included, the covariance matrix } for the selected population is: 


[so sc 


which we estimate using our selected sample. 


Under the assumptions of linearity and homoscedasticity it can be shown that: 


> - a pe eae 
pee ae Swi 7 Doi Dent Dal eden) Dae 


In effect, this correction reduces the observed sample variance of the outcome 
to what we would theoretically see if we were able to observe outcomes for the full 
population. As such, the full set of explanatory variables used in the most fully specified 
model should be included in &.. We include all six test scores and dummies for 
demographics, CSU or UC campus (where applicable), and CCI levels. Then in order to 
calculate the correlation matrix P, where 


P= (diag(3))*3(diag(3))” 


we delete the rows and columns from ¥ which contain the variances and covariances for 
explanatory variables not included in the model being estimated. For additional derivations 
and further discussion, see Lawley (1943); Lord, Novick, and Birnbaum (1968); and Lewis 
(2006). 


*2 Due to data considerations, we use the population of applicants to the CSU or UC as our full population. 
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